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Abstract 


According to the dominant view, time in perceptual decision making is used for 
integrating new sensory evidence. Based on a probabilistic framework, we investi¬ 
gated the alternative hypothesis that time is used for gradually rehning an internal 
estimate of uncertainty, that is to obtain an increasingly accurate approximation 
of the posterior distribution through collecting samples from it. In the context of 
a simple orientation estimation task, we analytically derived predictions of how 
humans should behave under the two hypotheses, and identihed the across-trial 
correlation between error and subjective uncertainty as a proper assay to distin¬ 
guish between them. Next, we developed a novel experimental paradigm that 
could be used to reliably measure these quantities, and tested the predictions de¬ 
rived from the two hypotheses. We found that in our task, humans show clear 
evidence that they use time mostly for probabilistic sampling and not for evidence 
integration. These results provide the first empirical support for iteratively im¬ 
proving probabilistic representations in perceptual decision making, and open the 
way to reinterpret the role of time in the cortical processing of complex sensory 
information. 


1 Introduction 

Making decisions is one of the most fundamental aspects of cognition and accordingly, its behav¬ 
ioral, neural and computational bases have been investigated extensively DHa. The prevailing 
framework of perceptual decision making maintains that time in decision making is used for col¬ 
lecting evidence about the stimulus for the decision ID. This proposal is corroborated by the fact 
that with the progression of time, the error of and the uncertainty about the decision in behavioral 
studies of decision making steadily decrease until they reach an asymptote presumably set by the 
internal noise of the system 0. These behavioral studies handle decision making typically as a de¬ 
terministic process and investigate the simplest two-altemative-forced-choice version of the problem 
yielding insights that are difficult to generalize to multiple options and estimation tasks (implying a 
continuum of options) abundant in real life |l6]|2l. 

In recent years, a prominent alternative framework of perception and cognition emerged based on the 
concept of probabilistic representations and computation 0. According to this view, humans and 
animals code not only a single value of the sensory input, but multiple values with their subjective 
uncertainty about those values, and thus representation and computation in the cortex is based on 
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probability distributions il. This proposal was supported by a number of studies finding that human 
and animal behavior in different tasks could be best described as if the participants were aware and 
used probabilistic representations to achieve their goals (approximately) optimally GoHni, and 
gained further support from recent neurophysiological findings 01 [la. However, the probabilistic 
framework aggravates the concern of generalization: the exact probabilistic inference necessary to 
reach a decision in a multiple-choice or estimation task is computationally clearly intractable, and 
the potential of appropriate approximations has not been explored to date. 

We selected one particular approximation strategy - Monte Carlo sampling - for probabilistic com¬ 
putation ijTbl, and explored the consequences of this framework on the role of time in decision mak¬ 
ing. Approximating probabilistic representations and computations by sampling means that time is 
needed to collect samples sequentially from the posterior distribution even when no new evidence 
enters the system and thus the posterior itself remains static. This provides a new hypothesis with a 
novel role of time in sensory systems that complements evidence integration: time is used for itera¬ 
tively refining an approximation of the true posterior. Thus, the goal of the present paper is twofold. 
First, to find evidence at the earliest possible stage of visual information processing for probabilistic 
computations with iterative approximation. Second, to establish whether evidence integration or 
probabilistic sampling dominates decision making in naturalistic estimation tasks. 

Our paper presents the following three new contributions. First, we established a mathematical 
model of decision making under evidence integration (El) and probabilistic sampling (PS), and 
by analytical derivation, we identified the measure of the trial-by-trial correlation between error 
and uncertainty as an appropriate measure to distinguish whether participants are predominantly 
occupied with El or PS (Section 2). Second, we developed a novel behavioral test method, which can 
be used for measuring immediate error and subjective uncertainty simultaneously on a trial-by-trial 
basis within ~700 msec after stimulus presentation (Section 3). Einally, we measured the correlation 
between error and subjective uncertainty in this modified orientation estimation task as a function 
of stimulus presentation time, and found that the pattern of this correlation confirms the predictions 
derived from the PS hypothesis (Section 4). Thus, we conclude that probabilistic computations take 
place from the earliest levels of cortical visual information processing, and that in simple perceptual 
decision making tasks with static stimuli the majority of time is used for improving an internal 
representation of the obtained information and not for incorporating new external evidence. We 
discuss our results in the broader context of perceptual decision making in Section 5. 

2 Theoretical predictions for evidence integration and probabilistic 
sampling 

In the Supplementary Information (SI) we derive anaWtical formulae for how estimation error, un¬ 
certainty, and their correlation should vary over tim^ in a perceptual (eg. orientation) estimation 
task under two orthogonal scenarios: 

Evidence integration (El) The posterior distribution (over possible orientations) is represented ex¬ 
actly at any moment, and time is used to collect more evidence about the external stimulus 
thus leading to a gradual refinement of the corresponding posterior, “homing in” on the true 
stimulus value (orientation). 

Probabilistic sampUng (PS) The posterior itself does not change in tim^ but it is represented 
approximately by drawing successive samples from it as in Monte Carlo techniques El, 
and so time is used to collect more samples from this static posterior. 

The assumptions underlying our derivations were the following: 

- The representation of uncertainty is self-consistent, in that uncertainty is predictive of the 
errors made in estimation (which we confirmed empirically, see below). 

- Across trials, the posterior changes its location and scale (mean and variance, which are 
both finite), but not its shape (higher moments). Importantly, while we show results here 

'“Time” here corresponds to “presentation time” in the experiments described in the following sections. 
^This can be justified by evidence integration converging to a finite-width posterior due to fundamental 
limits on perception, with convergence being fast because the stimulus is static. 
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Figure 1. Theoretical predictions. Normalized average error and uncertainty (top) and the cor¬ 
relation between error and uncertainty (bottom) for evidence integration (El, left) and probabilistic 
sampling (PS, right). Averages and correlations are computed across trials. Average error and uncer¬ 
tainty are normalised to their values obtained after the first data point for El, and to the asymptotic 
value in the limit of infinite samples for PS. (An above-zero asymptote for El can be obtained by 
introducing behavioural noise, see SI, or an asymptotically non-vanishing posterior variance.) Error- 
uncertainty correlation is normalised to its asymptotic value for both El and PS, where the asymp¬ 
totic value is obtained by using an exact representation of uncertainty, and is shown in the inset as 
a function of the coefficient of variation (CV) of the task difficulty (ie. the CV of the across-trial 
distribution of posterior variance). 


for a Gaussian posterior, the derivations themselves do not assume Gaussianity and work 
for any other shape with a known (and finite) kurtosis. 

- Successive data points (for El) or samples (for PS) are i.i.d. This means that the “number 
of data points / samples” referred to in the following should be understood as “the effective 
number of independent data points / samples” which is proportional to the number of data 
points / samples and inversely proportional to some suitable measure of their statistical 
dependency (e.g. the total autocorrelation of samples when computing the Monte Carlo 
integral of a simple linear function under the posterior). 

- Eor analytical tractability, there are two departures from the setting we tested experimen¬ 
tally: the variable over which the posterior needs to be represented is linear rather than 
circular, and estimation error is measured as squared rather than absolute error. We expect 
neither of these to impact our results qualitatively. 

The main results of our derivations are the following (Pig.[T]): 

- In both scenarios, the average error and uncertainty decrease roughly as 1 / time. This is 
interesting because a decrease in error with time has classically been taken as a hallmark 
of EL We show that it can equivalently result from PS, without El. 

- The time course of the correlation between error and uncertainty distinguishes between El 
and PS. In El, it remains constant (or decreases if behavioural noise is considered), while in 
PS it may potentially transiently decrease, but eventually always increases even well after 
average error and uncertainty have already asymptoted. 


3 Measuring trial-by-trial error and uncertainty in an orientation 
estimation task 

In order to asses whether human decision making is dominated by El or PS, we developed a new 
experimental paradigm (Pigure|^. The basis of our paradigm was a standard orientation estimation 


3 
















Fixation Point 




Figure 2. Experimental design. A. In each trial, the subject made an orientation estimation judge¬ 
ment and provided information about the orientation and their subjective uncertainty by drawing a 
single stroke on a tablet. See text for details. B. Estimation error (Ex) between the true and reported 
orientations, and level of uncertainty (cr^) were the dependent variables in the experiment. 


task. In each trial, subjects first saw a blank screen with a fixation dot for 1100 msec. Next, a display 
appeared with a variable number (1-6, randomly chosen) of 1-degree-long line segments equidis¬ 
tant (with a randomly chosen rotation) around a circle (extending 7° of visual angle in diameter). 
Contrast levels were sampled randomly (without replacement within a display) from {10, 20, 30... 
100%}, and orientations uniformly from 0°-180°. The display appeared for one of nine possible du¬ 
rations (“presentation time”): 50, 75, 100, 133, 167, 200, 300, 400, or 600 msec. After the display 
disappeared, a mask of random noise appeared with a small red circle identifying the position of one 
of the segments in the preceding display. The subject’s task was to report as quickly as they could 
their estimate of the orientation of the segment in the cued position simultaneously together with 
the subjective assessment of uncertainty in their estimate by drawing a line on a tablet with a stylus. 
The orientation of the line indicated the estimated orientation of the line segment, while the length 
of the line corresponded to subjective uncertainty. (A longer line indicated less certainty to avoid 
the possible confound in measuring error-uncertainty correlations as shorter segments provide in¬ 
herently less precision for orientation. ) After the subject responded, the mask and cue disappeared 
and a small segment appeared at the tested location with the orientation chosen by the subject and 
with a gray wedge around the line segment with a width (subtended angle) corresponding to the 
reported uncertainty. (The true orientation of the segment was not displayed.) This feedback display 
appeared for 500 msec, after which a new trial began. 

To enhance the quality of subjective uncertainty estimation, a scoring function was used to assess 
subjects’ performance on each trial. Subjects were instructed that their goal was to maximize their 
score which was calculated by combining the accuracy and certainty of their response. As a scoring 
function, we used the log probability of the true stimulus orientation under a circular Gaussian (von 
Mises) distribution defined by the subject’s response (segment orientation - mean, wedge width - 
concentration). This scoring function can be maximised if the subject’s uncertainty report reflects 
their true subjective uncertainty which in turn is predictive of the errors they are making Ha. To 
prevent subjects from developing simple feedback-based strategies while keeping them alert, sub¬ 
jects received only grouped feedback after every 10 trials in the form of an average score. Subjects 
completed 3-4 sessions of 900 trials across multiple days. To familiarize themselves with the pro¬ 
cedure and to facilitate the precision of mapping from uncertainty to line length, prior to each test 
session subjects had a practice session with 50 trials during which they received feedback after every 
trial including the true orientation of the cued segment. Data from these trials was not included in 
the analyses. 


4 Results 


We collected data from a total of iV = 5 subjects, four of whom were naive while the last one was 
informed about the goal of the experiment. We found no difference in performance between the 
naive and informed subjects confirming that the paradigm measured direct reactions of the subjects 
without much cognitive influence. 


4 













stimulus angle (deg) normalized x coordinate 


Figure 3. Control measures. The experiment gives veridical trial-by-trial information about sub¬ 
jects’ error and subjective uncertainty. A. The distributions of the test line segments’ true and re¬ 
ported orientation. B. Trial-by trial correspondence between the line segments’ true and reported 
orientation across all subjects and trials. C. Trajectories of strokes for all subjects normalized (ro¬ 
tated and scaled) such that they go from (0,0) to (1,0). 


4.1 Basic measures and controls 

First, we checked whether in our paradigm we measured the relevant aspects of human performance. 
In order to measure the typical pattern of trial-by-trial error and uncertainty, the stimuli must cover 
the entire space of orientation, the subject’s perception needs to follow the true stimuli, and re¬ 
sponse movements need to be ballistic. Figure confirms that these requirements were fulfilled. 
The stimulus distribution was nearly uniform in the space of orientations, and the subjects’ response 
was similar without any evidence for a bias in the cardinal directions (Fig 2A). Subjects’ judgement 
faithfully followed the true orientation of the target line segment (Pearson’s r=0.97, p<10“^) (Fig 
2B), and their stroke was a straight line with average deviation from the straight line between the 
starting and endpoints below 3.3±0.2% of the length of the stroke (Fig 2C). In addition, we calcu¬ 
lated the time profile of the strokes and found that subjects’ mean duration of drawing was 450±110 
msec (meanis.e.) with standard deviation of 160±30 msec. This suggests that subjects drew the 
line segments with a fast, single stroke without much fine-tuning, explicit cognitive deliberation, or 
modulation by different aspects of the task. 

4.2 The representation of error and uncertainty 

Next, we tested whether subjects’ uncertainty reports were predictive of their estimation errors. 
Figure 1^ shows a typical subject’s result with trials binned by reported uncertainty and the resulting 
error histograms fitted with a circular Gaussian. The error distribution remained centered at zero, 
but showed a clearly increasing spread as the subject’s subjective report of uncertainty about the 
correctness of the trial increased. 

Figurej^shows the same fitted circular Gaussians to each subject together with the underlying scatter 
plots of (absolute) error vs. reported uncertainty. Despite individual variations, each subject showed 
the same general relation of increasing uncertainty corresponding to steadily increasing error in 
their performance. This suggests that subjects had a reliable representation of the quality of their 
perceptual information and faithfully reported this through their stroke. Thus, our experimental 
paradigm and response method successfully captured subjects’ trial-by-trial error and uncertainty. 

4.3 The effect of task difficulty on error, uncertainty and their correlation 

We investigated how task difficulty affects subjects’ error level, uncertainty and the correlation be¬ 
tween the two. As task difficulty increased either by increasing the number of line segments in 
the display or by decreasing the contrast of the target segment, both the error rate and uncertainty 
of the judgment increased significantly (Figure 1st column, top two panels; the absolute value 
of Spearman’s p was between 0.60 and 0.95, with p<0.002 in all cases). Expressed in terms of 
reaction times (RTs), we found the same trend; as RTs increased, presumably due to finding the 
trial more difficult either because of the nature of the sensory input (more segments, less contrast) 
or for some other reason (e.g. lapse of attention, or focusing on an irrelevant part of the display), 
both subjects’ errors and level of uncertainty increased significantly (Figure]^ 1st column, bottom 
panel; Spearman’s p>0.87, with p<0.001 in both cases). Thus all three manipulations had a signif¬ 
icant modulatory effect on both error and uncertainty of subjects’ response. The effect of contrast 
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Figure 4. Relation between error and subjective uncertainty in a single subject. Signed ori¬ 
entation estimation errors fitted with a circular Gaussian in trials binned according to the reported 
uncertainty (labels on top). 
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Figure 5. Relation between error and subjective uncertainty across all subjects. Subjects’ 
orientation estimation error changed according to their subjective uncertainty. A. Circular Gaussian 
ht of orientation estimation error histograms corresponding to different levels of reported uncertainty 
(as in Fig. 0- Scatter plots of (absolute) error vs. uncertainty. Error bars show the mean±standard 
deviation for the data points in each uncertainty bin, gray lines represent linear regression fitted to 
the scatterplot. Note that it is the mean of absolute error plotted in panel B that is related to the 
standard deviation of signed error in panel A. 


is particularly remarkable, as each line segment in a display had a different contrast level. Thus, the 
fact that the contrast level of the target line segment had an effect indicates that subjects possessed 
a multivariate representation of uncertainty, whereby they represented their uncertainty for multiple 
objects in the scene individually, rather than having just a single summary measure of uncertainty 
about the whole scene. 

In contrast, task difficulty did not have an effect the predictive relationship between error and un¬ 
certainty. We quantihed the effect of task difficulty on the error-uncertainty regression (gray lines 
in Fig. 1^) by first selecting trials based on either number of line segments, contrast level, or RT 
duration for each subject, and then taking the slope and intercept of the regression line fitted to data 
collected in these trials (Figureright two columns). Neither the slope nor the intercept were sen¬ 
sitive significantly to changes in any of the three parameters of contrast, line number, or RT (p>0.24 
for Spearman’s correlation in all but one case, for which p>0.098, i.e. still non-significant). This 
means that while task difficulty reliably influenced subjects errors and uncertainty levels, it left the 
calibration between the two essentially unaltered, suggesting that subjects used a single, universal 
internal scale of uncertainty - in line with theories of probabilistic representations. 
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Figure 6. Average error and uncertainty depend on task difficulty, but not the calibration of 
uncertainty against error. Top row; dependencies on the number of line segments. Middle row; 
dependencies on the contrast level of the target segment. Bottom row; dependencies on reaction 
time. Left column; average error rates and uncertainty levels. Middle column; the slope of the 
error-uncertainty regression. Right column; the y-intercept of the error-uncertainty regression. 


4.4 Across-trial correlation between error and uncertainty as a function of stimulus 
presentation time 

To test the main prediction of the present paper, we analyzed error, uncertainty and the correlation 
between the two as a function of presentation time. Both subjects’ errors and their uncertainty de¬ 
creased significantly until around 133 msec (Figure]^, shaded area) and then remained constant for 
the rest of the duration of the trial (Figure]^, area without shading). Despite this coordinated decre¬ 
ment in error and uncertainty, the correlation between the two did not increase but instead decreased 
significantly, as the derivations overviewed in Section 2 predicted it both for El and PS (Figure]^, 
shaded area). However, immediately after error and uncertainty asymptotes in Figure]^, a rapid 
increase in the correlation began until the curve saturated after 400 msec of presentation time (Fig¬ 
ure]^). A one-way ANOVA including data from beyond the cutoff point showed that the increase 
in correlation was significant (p< 0.037) and specifically the correlation at time points at the trough 
of the dip (167-200 msec) were significantly lower than those at the plateau (300-400-600 msec) 
of the curve (p<0.0014). These results are in close match with the prediction of the PS model and 
cannot be explained by the El model. 

5 Discussion 

In this paper, we 

1. presented a new hypothesis about the role of time in perceptual decision making based on 
the idea of sequential approximate probabilistic computation in the cortex; 

2. derived models and analytical predictions as to how to detect behaviorally whether this 
proposed role or evidence integration dominates during a decision making task; 

3. developed a novel paradigm for testing the models’ predictions; and 

4. after conducting the experiment, we confirmed that, indeed, humans show the hallmarks of 
probabilistic sampling in perceptual decision making tasks. 

We envision two aspects of our contribution to be of significance for the field of computational 
neuroscience. Eirst, our paradigm involved orientation estimation, arguably one of the simplest 
perceptual decision making tasks, and our results showed that the uncertainty-error correlation is 
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Figure 7. Presentation time-dependence of the trial-by-trial correlation between error and 
uncertainty. The rising part of the correlation curve confirms probabilistic sampling during the task. 
A. Average error (red) and uncertainty (blue) curves as a function of presentation time. The shaded 
area represents the cutoff point (obtained independently for error and uncertainty, but coinciding 
at the same presentation time) above which there was no significant change in values between two 
neighboring points of the curves. Cutoff points were determined by recursively calculating the 
significance of the difference between correlations computed from data at a particular presentation 
time vs. data at all longer presentation times. Cutoff points were defined as the first time point (going 
backwards from the last time point) at which this comparison yielded a significant difference. B. 
Average correlation between error and uncertainty as a function of presentation time. The shaded 
area is inherited from panel A. Error bars in both panels represent s.e.m. 


not affected by dramatic changes in task difficulty, such as 80% contrast variation or 3-fold increase 
in the number of potential targets. This indicates that uncertainty is not an “add-on” cognitive 
metric that is assessed post-perceptually by a separate process, but rather an essential aspect of 
sensory representation that is inherently encoded together with the feature value of the input from 
the very beginning of the processing. This result provides a much needed extension of earlier studies 
proposing approximate probabilistic coding in the cortex by generalizing those claims down to the 
level of early sensory perception. 

Second, our new paradigm allows us to investigate the effect of sampling-based probabilistic coding 
with higher sensitivity than it was possible before. Earlier works explored the issue of sampling with 
methods that used probability matching as the assay for sampling-based probabilistic coding, and 
hence they could only hope to detect sampling when it is using one or very few samples HMD. 
This technical difficulty led to some confusions as to whether one can see evidence of sampling in 
particular tasks. With our method, the effect of sampling can still be tracked in the regime of a few 
hundred samples. Indeed, a rough estimate based on our results suggests that in the present task, 
samples might arrive as fast as one every couple of milliseconds. This number is feasible only if the 
cortex uses an overcomplete representation. Nevertheless, even this speed presents no insurmount¬ 
able problem to our experimental assay, opening the possibility of conducting experimental tests of 
probabilistic representations in a wide variety of contexts. 

It is important to clarify that even though PS has been introduced in this paper as an alternative 
to El, the two are not mutually exclusive and not even clearly separable in time. Sampling starts 
from the first moment evidence is obtained, and evidence integration does not “stop” at some point, 
only its benefit vanishes. Thus, an obvious follow-up in our research program is to analyze the 
relationship between evidence integration and probabilistic sampling in various tasks to shed light 
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on the common and separate aspects of the two processes. Furthermore, although we analyzed PS 
as a particular approximate inference method the cortex may employ, there could be other such 
candidates, and it will be interesting to see whether other methods that also iteratively refine the 
approximate posterior, e.g. (loopy) belief propagation or expectation propagation, result in similar 
temporal trends in the error-uncertainty correlation. 
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Supplementary Information 


1 Introduction 


We consider an estimation task, in which subjects have to estimate an environmental variable, a; € M 
(which we assume to be scalar for simplicity). In each trial, the subject receives partial information 
about the true value of x, x*, through making observation y and is asked to report two things: 

1. their best estimate of x* - assuming that they optimise a squared error penalty between this 
estimate and x *; 

2. their subjective uncertainty about this estimate, which we take to be their expected squared 
error between their estimate and x*. 

We will be interested in how subjective uncertainty, the actual squared error of the estimate, and the 
correlation between these two quantities change as a function of time (within in a trial) - where time 
is going to parametrise some key aspects of the model. 


2 Exact representation of uncertainty 


A complete representation of uncertainty means that the subject represents Px{x\y), a posterior 
distribution over x given the partial information available to them in y. First, in this section, we 
consider the case when the amount of information received about x does not change over the trial, 
ie. Px(a;|y) remains fixed, and the representation of uncertainty is exact, i.e. the relevant parameters 
of Px(a;|y) are directly and exactly accessible by the subject. This will then serve as the starting point 
for derivations regarding evidence integration (El, sections]^ andand probabilistic sampling (PS, 
sections EB- 

We introduce the following quantities to characterise the statistics of x under Px(a:| 2 /): 



x] 



(x - yxf 


II 

(x - px)^ 



(x - px)^ 

/cr^-3 


mean 

(1) 

variance 

(2) 

skewness 

(3) 

excess kurtosis 

(4) 


(As an example, ■Cx = ^x = 0 when Px(a:|t/) is a normal distribution.) 

Note that, in general, Px(a;|y) is going to be different on each trial using the same x* because the 
particular information given by the observation y is different. Thus, the moments of Px(a:|t/) will 
also be different on each trial, that is, importantly, they are random variables themselves. For sim¬ 
plicity, we will assume in the following that while px and change across trials, ^x and Kx remain 
constant - i.e. the posterior only changes in its location and scale but not in its shape otherwise. 


2.1 Task difficulty, 7, and properties of the posterior mean, and variance 

An exact (and consistent) representation of uncertainty means that the true value of x, x*, behaves 
as if it was actually sampled from Px(x|t/). To simplify notation, however, and without loss of 
generality, we will assume that on each trial x* = 0 and shift the reference frame for x accordingly. 

In order to minimise squared error, the best estimate of x* the subject can report is their posterior 
mean, Remember (see above), that y^ is itself a random variable. In fact, given our shift of 
reference frame, the distribution of y^ around x* = 0 is the mirror image of the distribution of x 
around y^, ie. P{yx = x') = Px(ffx — x'). 
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We define the variance of /Xx as the (squared) difficulty of a trial, 7 . This implies that the mean, 
variance, and fourth (central) moment of /Xx are 


E/2x|7[Mx] 

= x* =0 

(5) 

E/ixh [l^x] 

= 72 

( 6 ) 

Emx|7 [/^x] 

= (kx + 3) 7 "^ 

(7) 


where we have made use of the fact that the distribution of ^x is the ‘mirror image’ of the posterior 
(see above) and that it is centred on x* =0 and thus its variance is equal its mean squared error. 

Note that, in general, 7 itself might also change from trial-to-trial, as some trials might be more or 
less difficult, or we may be paying more or less attention to the stimuli, so we will treat it also as a 


random variable with the following statistics: 

( 8 ) 

V^[7]=ct2 (9) 

E7 [7^] = ^^7= =^7+0-^ (10) 

V^[72]=a2. (11) 


It will also be useful later to define the squared coefficient of variation of 7 ^: = cr^ 2 //r ^2 > 0. 

A consistent representation of subjective uncertainty should be predictive of the errors the subject 
makes. Ideally thus, in the above estimation task, the reported uncertainty should be the variance 


of the posterior, which will be equal to the variance of /Xx, which in turn by definition is 7 ^. 

21 2 

CTxIt = 7 

which in turn entails that the following is true for any k: 

E.XI 7 K] =7^= 

( 12 ) 

(13) 


It is also useful to note that the same equality (ie. that the distribution of cr^ given 7 is 
distribution) also implies that and p.x are independent given 7 . 

2.2 Some useful identities 

We use the formula Ex [/(A')] = Ey [Ex|y [/(AT)]] to derive the following. 

a delta 


E^x [Mx] 

— E.y [E^^|..j,[^x]] 

= E^[0] 

= 0 
(14) 


E/ 2 x [A^x] 

= E^ [E;2x|7 [A^x]] 

= E7M 

1^72 

(15) 


E^x[M^] 

= E^ [E;2x|7 [A^x]] 

= Et,[(kx + 3) 7'‘_ 

= (Kx 

(16) 

+ 8 ) (/X ^2 + (7^2 


= E^ [Ecr „|7 [^x]] 

= E7M 

= /X.y2 

(17) 



= E,[E.^|.,[a4]] 

= E7[7l 

(18) 

+ cr ^2 

E^x.-Tx [MxC^x] 

= E7[E^x.cTx|7[A^xCr3] =E7 [E;Xx|7[a^x] 

= E^, [ 7 ^ 7^] 

= (4^ 

(19) 

+ Cr ^2 

E/ix.CTx [Mx 


= Et, [ 07 ^] 

= 0 
( 20 ) 



2.3 Performance 

The average squared error for an exact representation of uncertainty is simply the squared error of 
the mean that is 

e = [Mx] = A^72 (21) 
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2.4 Correlation between error and uncertainty 

Now, we can ask the question how much the error in the estimate is going to correlate with the 
subjective representation of uncertainty. For this we first compute the covariance of these two quan¬ 
tities: 


[Mx: [Mx [Mx] E^x [^l] = -f cr^2 - /iT,2 ^J,^2 = a^2 (22) 


(23) 


Likewise, we can compute the respective variances: 


^Mx [Mx] — E/ 2 x [^^t] ~ (E^x [Fx]) — (^x + 3) (p^2 -f 0 -^ 2 ) — fij2 — (Kx -f 3) (7^2 -f (Kx + 2) ^^2 

(24) 


_[al]=E^^[at]-{E^^[al]Y =/r^2 + ^ -/ri 


(25) 


So the correlation between squared error and subjective uncertainty is 

P = 


P = 


^^(ttx + 3) (7^2 + (ttx + 2) /r^2^ 
1 

W^/tx + 3 -f (kx + 2) uj^2 


(26) 

(27) 


There are three things to note about this result: 

1. The correlation is not going to be 1 in the general case, even though we are talking about 
an exact representation of uncertainty. How can that be? This is because only the expected 
value of the squared error, not the squared error itself is predicted by uncertainty, otherwise 
we could be cheating (i.e. if we could predict the error itself, than we might as well subtract 
it from our estimate thus making our error zero)p^ 

2. The correlation is going to be zero when a ;^2 —>^ 0, so we need trials with varying difficulty 

to get meaningful results. (This is was expected intuitively, as uj ‘^2 0 means that all 

trials have the same difficulty, and associated posterior variance, and therefore the same 
uncertainty estimate - and one obtains a trivial zero correlation between two quantities one 
of which does not ever change.) 

3. Nothing changes with time here - simply because we ascribed no role for time in the model. 


3 Evidence integration 

We now consider the case of a probabilistic representation that is still exact, as before, but it is 
representing a posterior distribution that is changing in time because it is conditioned on increasing 
amounts of data, Px(x|j/i... 2 /„). 

In general, the variance of such a posterior scales inversely with n, the number of data points. Using 
the equivalence of difficulty, 7 ^ and posterior variance cr^ described in the previous section, this 
means that the “effective difficulty” of trials in which n data points have been observed can be 
expressed as 

7 n = (28) 

"‘Well, we wouldn’t quite be cheating in a special case: when the error is known only up to its sign. In that 
case, the squared error is known exactly but we still can’t subtract away our error because we don’t know its 
sign. Indeed, in that case the distribution of fix is a mixture of two delta distributions (with equal weights) 
sitting at equal distances from zero, and thus Kx = —2, in which case the formula in Eq.|27|correctly predicts 

p = 1. 


12 







where we use 7 ^ here to describe the difficulty of trials with a single data point. In turn, this means 
that the relevant moments characterising the distribution of 7 ^ scale as 

IJi^2 = (29) 

cr^2 = cr^2 jn^ (30) 

Note that this scaling implies that the coefficient of variation, which ultimately matters for the anal¬ 
ysis of error-variance correlations remains constant, and so 

= W^2 (31) 


Beside the variance, the (excess) kurtosis of V^{x\yi .. .yn) is also expected to change and converge 
towards zero, as the posterior tends to converge to a normal distribution with increasing amounts of 
data (modulo degenaracies) - this is the argument on which the usual Laplace approximation is 
based. We will make the assumptiorj|^that the kurtosis of the distribution also scales as 1/n 

Kn = K^/n (32) 

where we (re-)use to denote the kurtosis of the posterior after observing a single data point. 


Putting the above scalings together and substituting them to the results of the previous section, we 
can now derive the evolution of estimation errors and error-uncertainty correlations in the context of 
evidence integration as the following: 


e = 


n 


P = 


1 


^K^/n - 1 - 3 - 1 - {kx/it- + 2 ) 


(33) 

(34) 


4 Approximate representations of uncertainty 

We now return to the case of a static posterior distribution. In an approximate representation of 
uncertainty, the posterior mean and variance are not directly accessible, only their approximate 
estimates, /lx and respectively, which can be noisy and / or biased. 


In this case, the performance and correlation between the reported uncertainty and actual squared 
error can be computed as 


e 

P 

where 

— iEax [Ax] 

CAx.^x[Ax.O-x] 

x/Vax[A^] 

— Emx.O-x [®Ax|Mx,0-x [Ax]] 

(35) 

(36) 


= Ea,,[A^] -(KaJa^)' 

= IEa1x,0-x [®Ax|Mx,0-x [Ax]] ~ (^/Jx 

,.x[EAx|.x..x[Aa])' 

(37) 



= IEa1x,0-x [®CTx|/2x.O-x ['^x]] ~ (®Mx. 

.x[E.x|.x,.x[^^]])' 

(38) 

^Ax.ITx [Ax> '^x] 

= EAx,0-x [Ax '^x] ~ ®Ax [Ax] ®0-x ['^x] 

= E/ixiO-x [^Ax.CTxIMx.O-x [Ax ^x]] ~ 
~ E/Xx.CTx [^AxIMx.CTx [Ax] ] ®Atx,0’: 

X [Eo-xIMx.O-x ['^x]] 

(39) 

Therefore, in order to be able to compute these quantities of interest, we need to know the following 
five statistics of the approximate representation: 


^AxIMxjCTx [Ax] 

EAx|Mx,<Tx[Ax] 

(40) 


®CTx|aIx.O-x ['^x] 

E<Tx|/Xx.<Tx [o-x] 

EAx.O'xI/^x.O'x [Ax '^x] 

(41) 

(42) 


‘''See Appendix. 
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A subtle note about approximate representations: remember, that the posterior variance was only 
relevant for the subject in the context of the estimation task inasmuch as it predicted the average 
squared error between the posterior mean and the true stimulus value - which was the case for an 
exact representation of uncertainty. However, for an approximate representation of uncertainty, the 
total estimation error is compounded by the fact that the posterior mean itself is also only estimated. 
Thus, ideally, the subject should estimate their own estimation error of the posterior mean, based on 
their estimate of the posterior mean and variance, and add it to the estimate of the posterior variance, 
and report this total error estimate, as their uncertainty. We will return to this point later in the 


context of sampling-based representations in section 5.4 


5 Sampling-based representations 

We now consider the case of sampling-based representations, that is when we do not have direct 
access to the relevant parameters of the posterior, Px(a:|y), such as /ix and cr^, but only to a finite 
number, N, of samples Xi, i = 1... N, that are iid. according to Px(a:|?/). We will start by con¬ 
sidering some statistical properties of these samples that will be useful for the derivations later. We 
will then derive unbiased estimators that estimate the mean and variance of the posterior from the 
samples. Next, as a side, we will analyse the basic statistical properties of these estimators, namely 
their variances and covariance (and correlations). Then, we will derive the correct measure of uncer¬ 
tainty in the context of a sampling based estimator, that takes into account its own estimation errors, 
and derive the statistical properties of the mean and uncertainty estimate that are directly relevant 
for computing the performance and error-variance correlation. Finally, by combining results from 
previous sections we will derive how performance and error-variance changes with the number of 
samples. 

5.1 Statistics of the samples 

The fact that a;^, i = 1... TV are identically distributed according to P^{x\y) of course entails that 
the mean and variance (and other moments) of any individual sample will match the corresponding 
moments of the posterior such that 



IE 3 ; [xi 

— Mx 



V 3 ; [Xi 



E:, 

Mx) 

= E^ 

{x - fj,^f 

E:, 

Mx) 

= E^ 

{x - 


(43) 

(44) 

= (45) 

= («x + 3) (46) 


5.2 Unbiased estimators of the mean and variance of the posterior 

We will use standard unbiased estimators (and their extensions) for estimating the posterior mean 
and variance from the iid. samples. (As a consequence, the squared error of these estimators will be 
their variance, so we will use these terms interchangeably in this context.) From here on, to simplify 
the algebra, we will assume (without loss of generality) that px = 0 - unless otherwise noted. (Note 
that this does not imply /tx = 0 .) 

The unbiased estimator for the mean of the posterior is simply the sample mean: 

1 ^ 

Ax = ^ X! (47) 

and the unbiased estimator for the variance is the sample variance: 

<^x = E (E^? (48) 
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5.3 Basic statistics of the sample mean and variance 
5.3.1 Variance of the sample mean 


Va;[/tx] = Ea;[Ax] “ (Ea;[/tx])^ 
~ 


_ 1 
“ W 

1 2 
= N^- 


N N 

i=l j=lj^i 


5.3.2 Variance of the sample variance 

In the following, we will make use of the following identities - assuming /Xx = 0: 

N N 

^ ^ Ej, [xf x|] = N {k^ + 3) a^ + N {N - 1) 

j=i 

N N N 

^ ^ ^ E^ [xf Xj Xk] = N {k^ + 3) a* + N {N - 1) 

i—1 j—1 k—1 
N N N N 

^ ^ ^ ^ E^[xi Xj Xk xi] = iV (kx + 3) + 3N {N - 1) 

i—1 j—1 k—1 1—1 


(49) 

(50) 

(51) 

(52) 


(53) 

(54) 

(55) 


And so the variance of the sample variance is (omitting several lines of tedious but trivial algebra) 

41 /ttt. r ^ 91 \ 2 


V4a2]=E4a4]-(E4a2])^ 

"kx 2 
iV A^- 1 


(56) 

(57) 


5.3.3 Covariance between the sample mean and the sample variance 

In the following, we will make use of the following identities: 


N N 


i — 1 j — 1 

(58) 

N N N 

i—1 j — 1 k—1 

(59) 


So the covariance between the sample mean and sample variance is (again omitting several lines of 
tedious but trivial algebra) 


[Ax) 


Ea: [Ax - Ea[Ax] Ea [<T^] 

Cx 3 

N ^ 


(60) 

(61) 


Note that this covariance is 0 if the posterior is normal, or in general when it is symmetric (because 

Cx = 0). 
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Although we don’t need this in the following, but it may be useful to note that the correlation between 
the sample mean and variance is thus 


Corr^; [/ix,o-x 



(62) 

(63) 


5.4 An improved estimator of uncertainty 

Remember that the objective of a representation of uncertainty is that it should be predictive of the 
squared error of one’s estimate. In a sampling based representation the best estimator (in a squared 
loss sense) is the sample mean, p-x- The error of this estimator is 

[Ax] = [Px] + ^ '^x (64) 

Of course, we don’t have direct access to cr^, but we have an unbiased estimator for it, using 
which we can derive the improved estimator of uncertainty as 

o-x = (l + (65) 


5.5 Statistical properties relevant for performance and correlation analysis 
5.5.1 Statistics of the sample mean 

In this section, we will make use of the following identities - without assuming px = 0: 


'Rx[xi]=iJ.x ( 66 ) 

Ex [xf] = cTx + Px (67) 

Ex [xf\ = + SpxCT^ + Px (68) 

Ex [xf\ = {kx + 3) (Tx + 4^xMxCTx + 6px(Jx + Px (69) 


Now we can derive the relevant statistics of the sample mean (once again omitting several lines of 
tedious but trivial algebra): 


and 


^ N N 

Ex [Ax] = ^x[Xi Xj] 

(70) 

i=l j=l 


2,1 2 
— A^x + ^ '^x 

(71) 

^ N N N N 

yZ yZ Ea;[a;j Xj Xk xi] 

(72) 

(73) 

i—1 j — 1 k—1 1 — 1 


4,6 22, 4^ Si'^xT 3A^ 4 

Mx + ^ l^x^x + J^2 SxMxCTx + ^3 cr^ 

(74) 


5.5.2 Statistics of the improved uncertainty estimator 




Et 





2 




N -I 


(75) 

(76) 
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p = 


5.5.3 Joint statistics of the sample mean and the improved uncertainty estimator 

(We once again omit several lines of tedious but trivial algebra.) 

= ((iV ^ 


(77) 

(78) 


5.6 Performance and error-uncertainty correlation with a sampling-hased representation 

Note that the quantities 

E, [A"1(Eqs. |70l7T] l EjA^1(Eqs. |7l74 i 

E4a2](Eq.|^ E4a4](Eq.|^ 

EjA;^^1(Eqs. |7TO 


derived in the previous section are exactly the quantities (using slightly different notation as required 
by the different contexts) 


®Ax|mx 


E, 


SxIM: 


M] 


%x|Mx,<Tx[Ax] 
E<5rx|Mx.<Tx [^x] 


E, 


j.CTx [a 


2 ;i 2 ] 

cr„ 


that are required to compute the error and error-variance correlation of sampling as an approximate 
representation in section]^ 

So, by making the appropriate substitutions (also from section [Z2] ), we can derive the mean squared 
error of a sampling based representation as 

e = E/ix [Ax] 


- ( 1 + ]V ^ 


(79) 

(80) 

(81) 

(82) 


Similarly, the variance of the error and the uncertainty are: 

^Ax [Ax] = E/^xiO-x [EAxI/^x.CTx [Ax]] ~ (Emx.O-x [EAxl/ix.O-x [Ax]] ) 

And so, finally, the error-variance correlation is 

^ + W + ^ (^^ 2 ^ + l) 

^Kx + 3-1- (kx + 2) uj^2 + ;^ ^6 + 4 u}^2 ^ + ;^ ^3 + 2 10^2 '^ + iv^) ^ 

(83) 


The following are worth noting about these results: 

1. The squared error, e, decays as 1/iV to the squared error of an exact representation of 
uncertainty, ^^ 2 . 

2. The behaviour of the correlation p is more complicated and depends on two quantities: 
uj^ 2 , quantifying how variable is the difficulty of trials across the experiment, and Kx, the 
(excess) kurtosis of the posterior. 

As a brief summary of the time-dependence of p, we can note the following: 
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1. The correlation, p is not always monotonically increasing as a function of the number of 
samples, N. In particular, it can be a (transiently) decreasing function of N when the 
following two conditions are satisfied; 

(a) N is small, 

(b) the kurtosis of the posterior, Kx is greater than zero. 

2. Importantly, the transient decrease in p is shorter for higher values of uj ^2 (darker blues), 
that is, if trials are sufficiently variable in terms of their difficulty, 00^2 > 1. In this case, 
the transient is over by the time performance, measured by e, converges. 

3. After this transient, p is indeed a monotonically increasing function of N and converges 
from below to the correlation exhibited by an exact representation of uncertainty. 

4. The time constant of the convergence of p can be substantially longer than the time constant 
of the convergence of e, in particular for higher values of and lower values of 10 ,^ 2 . This 
means that the in general, p will continue to increase when e is not discernibly changing 
anymore. 

Note that each of these features seem to be borne out in the data: 

1. There is a transient decrease in p. 

2. This transient is over by the time e stabilises. 

3. After this transient, p increases. 

4. This increase continues beyond the time when e stabilises. 

6 Behavioural noise 

One more important factor may influence the results derived so far: behavioural noise may corrupt 
the reported posterior mean, jl, and variance, This is particularly relevant for evidence integra¬ 
tion, where error is predicted to asymptote at zero (see above) which is clearly unrealistic. As we 
will see below, behavioural noise fixes this problem by yielding an above-zero asymptote for error. 

We will assume that the behavioural noise in reporting these quantities is unbiased and independent 
(from each other as well as from 7 ) and has variance e? and e? 2 , respectively, and derive these effects 
for an exact representation (useful for the static case as well as for the case of evidence integration), 
and for a sampling-based representation. 

In this case, the relevant quantities are 


V[A] 

= V[Ax] 

+ 4 

(84) 

V[A^] 


+ (Kp -f 2 ) + 4e? V[Ax] 

(85) 

V[a^] 

= V[a^] 

+ 4^ 

(86) 


= c[Ax, 


(87) 




(88) 


6.1 Exact representation 

The argument above gives the following formulae for the squared error and error-variance correlation 
in the static case: 


e — p ,^2 + 
P = 


+ 3) (7^2 + (kx + 2) + {k/1 + 2) + 4 e? p,^2^ (^a^2 + £^ 2 ^ 

1 

^Kx + 3-1- (kx + 2) + {Kp + 2) u }^2 /+ 4 e? ) (^ + 


(89) 

(90) 

(91) 
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For the case of evidence integration, we obtain: 
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A Convergence of the posterior kurtosis 


In this section our aim is to derive, in the context of the evidence integration model, how the kurtosis 
of the posterior, converges as the number of data points, n, increases, in particular what is the 
leading order (in n~^) of this convergence (at least asymptotically, in the n —> oo limit). 

Based on m, the rth moment of the rescaled posterior distribution (of x = b (x — can be 
written as 


K 

E 4 x^] = Xrj n-^/2 + (94) 

j=r 


where, in general, the constants depend on the particular data points on which the posterior is 
conditioned but all Xrj for which j is odd are zero, and for even r, the leading term of the sum can 
be given directly as 


which for 
yields 


A,, = 2’’/2r((r + l)/2) r-i(l/2) 

r = 2 
X22 = 1 


(95) 

and r = 4 (96) 

and X44 = 3 (97) 


Applying Eqs.|^and|^to the case of the hrst four moments of the rescaled posterior, and expand¬ 
ing the hrst couple of terms for each, we obtain 


7 


7 = 1 

(98) 

J 

= Ai2 n~^ + Ai 4 -f Aie n~^ + 

(99) 

j=2 

(100) 

= n~^ + X 24 n~^ + A 26 n~^ + 

(101) 

3=3 

(102) 

= A 34 n ^ + A 36 n ^ 0 (n 

(103) 

i=4 

(104) 

= 3n"^ + A 46 n~^ + 0(n~'^) 

(105) 


''The exact values of a and b will be unimportant later, but more specifically, a is the maximum likelihood 
estimate of x, and b is related to the inverse score differential at the maximum likelihood estimate. 
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The second and fourth central moments of the rescaled posterior can then be written as 


E, 


(a:-E^[a;]) J = E^ [a;^] - (Ea;[x]) (106) 

= n~^ + A 24 n“^ + A26 n~^ - [A12 n~^ + \un~'^ + Aie + 0 (n~*) 

(107) 

= n~^ + A24 + A26 n~^ - [XI2 n~‘^ + 2 A12 A14 n~^] + 0 {n~‘^ 


+ (A24 — X12) TJ ^ + (A26 — 2 Ai2 A14) n ^ (D(ti 


A 


22 


A 


23 


E, 


(x-E,[a;])^J =E4 s 4] _4E,[a;] E^S^] + g ^_2^ _ 3 

= 3n“^ + A46 n~^ — 

— 4 [Ai 2 74 ^ + Ai 4 n ^ + Ai6 n [A 34 n ^ + A 36 n ^] + 

+ 6 [Ai2 74 ^ + Ai 4 74 ^ + Aig 74 [ 74 ^+ A 24 74 ^ + A 26 74 — 

- 3 [Ai 2 74“^ + Ai 4 74“^ + Ai 6 74“^]"^ + 0 {n~'^) 

= 3 74 “^ + A 46 n~^ — 4 Ai 2 A 34 74 “^ + 6 \i 2 n~^ + O {n~*) 

= 3 74 ^ + (A 46 — 4 Ai 2 A 34 + 6 A^ 2 ) ^ ^ + C7 (74 


(108) 

(109) 

( 110 ) 


( 111 ) 

( 112 ) 


A, 


33 


From these, the second and fourth central moments of the original posterior can simply be written 
as 

1 


Ex 

{x - Mx)^ 

II 

Ex 

1 

II 


(a;-E2;[a;]) = {n ^ + A22 74 ^ + A23 n ^)+C >(74 


The kurtosis of the posterior can then be written as 


(a;-E^[a;]) =—(374 ^ + A33r4 ^)+e>(74 '‘) 


E, 


Kn = 


{x - 


-3 


(Ex {x - ^ 

3 74“^ + A 33 74“^ + O ( 74 “"^) 

(74-1 + A22 74“2 + A23 74 “ 3 ) ^ + C)( 74 “ 4 ) 
3 74“^ + A33 74“^ + 0 ( 74 “"^) 


-3 


74 “^ + 2 A22 74 “^ + 0(74“^) 

Expanding the first term, which is a rational function, in 74 “^, one obtains 
374"2 + A33 74"3 +0(74"^) , _2. 

^^^ = 00 +at 74 ^+ 0(74 


,-2 


+ 2 A 22 74-3+0(74-4) 


374“^ + A 33 74 “^ +0(74“^) = [oo +ai 74 “ 4 ] + 2 A22 74 “^] + 0 ( 74 “‘*) 

= oq 74“^ + 2 ao A22 74“^ + ai n ~^ + 0(74“^) 

= ao 74“^ + (2 ao A22 + ai) n~^ + 0(74“'*) 

from which one can derive, by matching terms of the same order on the two sides, that 

ao = 3 

ai = A33 — 2 ao A22 = A33 — 6 A22 
which means that k„ can be written as 

= 3 + (A33 - 6A22) 74“4 - 3 + 0 {n~‘^) 

= (A33 - 6 A22) 74“4 + 0 {n~‘^) 


(113) 

(114) 

(115) 

(116) 

(117) 

(118) 

(119) 

(120) 
(121) 

( 122 ) 

(123) 

(124) 

(125) 
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Thus, we have just derived that the kurtosis indeed goes to zero (because in the end, the zeroth order 
term in the expansion of is zero), and asymptotically converges as1 /n (because the term in the 

we have also proven the 


113 


expansion that is first order in n ^ is non-zero). (On the way, in Eq. 
convergence of the posterior variance, and thus 7 ^, also scales asymptotically as 1/n.) 
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